autovi: Automated Assessment of Residual Plots Using Computer Vision

ECSSC2024

Weihao (Patrick) Li

Monash University

🤔Challenges

  • Vertical spread of the points varies with the fitted values indicates the existence of heteroskedasticity.

  • However, this is an over-interpretation.

  • The visual pattern is caused by a skewed distribution of the predictor.

autovi Package

The autovi package provides automated assessment of residual plot with computer vision models.

It estimates visual signal strength (VSS), which is the KL divergence between a residual plot and a reference null residual plot.

Core Methods

  • Null residuals simulation: rotate_resid()
  • Visual signal strength: vss()
  • Comprehensive checks: check() and summary_plot()

💡Example: Boston Housing

fitted_model <- lm(MEDV ~ RM + LSTAT + PTRATIO, data = housing)
ggplot() +
  geom_point(aes(fitted(fitted_model), 
                 resid(fitted_model))) +
  theme_void()

rotate_resid()

Null residuals are simulated from the fitted model assuming it is correctly specified.

checker <- residual_checker(fitted_model)
checker$rotate_resid()
# A tibble: 489 × 2
   .fitted   .resid
     <dbl>    <dbl>
 1 632372.   -3870.
 2 525177. -145487.
 3 646753.    5602.
 4 624848.  122366.
 5 611817.  -12470.
 6 551051.  -45186.
 7 504757. -144455.
 8 445700.   70620.
 9 281912.   26909.
10 453398.  -86980.
# ℹ 479 more rows
checker$rotate_resid() |>
  checker$plot_resid()

vss()

Visual signal strength of the actual residual plot

checker$vss()
✔ Predict visual signal strength for 1 image.
# A tibble: 1 × 1
    vss
  <dbl>
1  6.48

Visual signal strength of a null plot

checker$rotate_resid() |>
  checker$vss()
✔ Predict visual signal strength for 1 image.
# A tibble: 1 × 1
    vss
  <dbl>
1  1.24

check()

checker$check()
── <AUTO_VI object>
Status:
 - Fitted model: lm
 - Keras model: (None, 32, 32, 3) + (None, 5) -> (None, 1)
    - Output node index: 1
 - Result:
    - Observed visual signal strength: 6.484 (p-value = 0)
    - Null visual signal strength: [100 draws]
       - Mean: 1.169
       - Quantiles: 
          ╔══════════════════════════════════════════╗
          ║  25%   50%   75%   80%   90%   95%   99% ║
          ║1.037 1.120 1.231 1.247 1.421 1.528 1.993 ║
          ╚══════════════════════════════════════════╝
    - Bootstrapped visual signal strength: [100 draws]
       - Mean: 6.28 (p-value = 0)
       - Quantiles: 
          ╔══════════════════════════════════════════╗
          ║  25%   50%   75%   80%   90%   95%   99% ║
          ║5.960 6.267 6.614 6.693 6.891 7.112 7.217 ║
          ╚══════════════════════════════════════════╝
    - Likelihood ratio: 0.7064 (boot) / 0 (null) = Extremely large

summary_plot()

checker$summary_plot()

💡Example: Left-triangle

Breusch–Pagan test \(p\)-value = 0.0457

💡Example: Dinosaur

Ramsey Regression Equation Specification Error test \(p\)-value = 0.742

Breusch–Pagan test \(p\)-value = 0.36

Shapiro-Wilk test \(p\)-value = 9.21e-05

🌐Shiny Application

Don’t want to install TensorFlow?

Try our shiny web application: https://autoviweb.patrickli.org

Thanks! Any questions?


tengmcing

patrick.li@monash.edu

📦 autovi

📜 Slides

Slides URL: https://ecssc2024.patrickli.org